Mobile application for signage design solution using augmented reality

ABSTRACT

A method of generating an augmented reality (AR) signage solution including acquiring an image of a structure, establishing scaling information for the image by correlating dimensional information of the structure to points in the image, obtaining jurisdictional signage rules, designing a layout for signage on the structure in compliance with the jurisdictional signage rules, depicting in AR the acquired image and signage layout, and generating production documentation after obtaining user approval. The method including detecting the location of one or more obstructions on the structure and incorporating the obstruction locations in the signage layout. The method including adjusting the orientation of the acquired structure image within the AR to reduce skew of the structure in relation to a viewing perspective. A system and non-transitory computer-readable medium are also disclosed.

CLAIM OF PRIORITY

This patent application claims, under 35 U.S.C. § 119(e), the prioritybenefit of U.S. Provisional Patent Application Ser. No. 62/647,603,filed Mar. 23, 2018, titled “Mobile Application for Signage Design UsingAugmented Reality” the entire disclosure of which is incorporated hereinby reference.

BACKGROUND

Conventional approaches for designing and selling signage can betime-consuming and error-prone, resulting in inefficiencies and loss tosignage vendors. Some customers might not fully appreciate theas-installed appearance or placement of signage until after itsmanufacture, purchase, and installation. There is a need for improvedmethods and systems for designing, implementing, and simulating theplacement of signage upon buildings or structures that can solve some orall of the aforementioned problems.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 depicts a system for implementing an augmented reality signageapplication in accordance with embodiments;

FIG. 2 depicts a process for generating an augmented reality signageimage in accordance with embodiments; and

FIG. 3 depicts an augmented reality visualization in accordance withembodiments.

DETAILED DESCRIPTION

Embodying systems and methods provide for the design and augmentedreality visualization of signage on the exterior of a building. Systemsand methods can be embodied as a mobile signage application on a mobilecomputing device (e.g., tablet, mobile phone, portable computer,dedicated tool, or other device). Implemented as a mobile app, embodyingsystems and methods can provide a remote site signage design tool. Inaccordance with embodiments, multiple individual signage designs can beaggregated as part of an augmented reality visualization. This augmentedreality visualization is superimposed on the building or structure sothat the overall signage presentation for a site can be visuallypresented for evaluation.

The terms “building”, “structure”, and “building/structure” as usedherein refer to the object upon which the signage is to be installed.Within this disclosure, these terms are used interchangeably withoutdistinction.

In accordance with embodiments, the signage application can activate acamera, imaging device, and/or other sensor on the mobile computingdevice. The signage application can be remote or local to the mobilecomputing device. The imaging device is used to gather information onthe building or structure upon which the signage will be designed andplaced. By way of overview, the signage application can capturephotographic or video images of a building or other object (exteriorand/or interior). Location information on the building/structure can beobtained by the signage application accessing a position location systemof the mobile computing device (e.g., global positioning system (GPS),or other location service). This positional information can be enhancedby utilizing available cellular communication system or wirelessinternet (Wi-Fi) networks.

In an exemplary embodiment, the captured imaging may form the backgroundfor an augmented reality (AR) display on the mobile device. This AR maybe of a two-dimension or 3D form. The background of an AR display whichshows the object, will form the backdrop upon which an image of thesignage to be designed, will be superimposed.

FIG. 1 depicts system 100 for implementing an augmented reality signageapplication in accordance with embodiments. An embodying system includesmobile computing device 110, which can include control processor 111that communicates with other components of the mobile device overinternal data/control bus 112. Control processor 111 accesses executableinstructions 113, which can be stored in memory 114 to implement signageapplication 115. The control processor may be a processing unit, a fieldprogrammable gate array, discrete analog circuitry, digital circuitry,an application specific integrated circuit, a digital signal processor,a reduced instruction set computer processor, etc.

The mobile computing device can include input/output (I/O) unit 119 thatcommunicates across electronic communication network 170. I/O unit 119can support one or more hardwire and/or wireless communication protocolsincluding, but not limited to, cellular telephone protocol, Bluetooth,low energy Bluetooth, iBeacon, near field communication, Airplay, etc.

Mobile computing device 110 can include imaging device 117 (e.g., acamera or other image capture device). An image captured by imagingdevice 117 can be displayed on display screen 118. The captured imagecan be a still photograph or a video. Laser measurement tool 120 canprovide accurate dimensional information on the building/structure. Thelaser tool can be internal to the mobile computing device, a plug andplay component accessible through a port, or a standalone unit thatprovides data to the mobile computing device.

Position location system interface 116 receives signals and determinesthe position of the mobile device based on the received signals. Thepositional location of the building/structure for which the sign isbeing designed can be located using the information from positionallocation system interface 116. This information can be used to obtaindimensional information on the layout of the structure (width, depth,architectural footprint).

Electronic communication network 170 can be, can comprise, or can bepart of, a private internet protocol (IP) network, the Internet, anintegrated services digital network (ISDN), integrated services digitalnetwork (ISDN), a modem connected to a phone line, a public switchedtelephone network (PSTN), a public or private data network, a local areanetwork (LAN), a metropolitan area network (MAN), a wide area network(WAN), a wireline or wireless network, a local, regional, or globalcommunication network, an enterprise intranet, any combination of thepreceding, and/or any other suitable communication infrastructure. Itshould be recognized that techniques and systems disclosed herein arenot limited by the nature of electronic communication network 170.

System 100 can include remote server 140 in communication with datastore 150. The remote server can be in direct communication with thedata store, or in indirect communication across electronic communicationnetwork 170. Processor unit 142 can execute executable instructions 152,which cause the processor to perform operations including, but notlimited to, signage design, specification, layout, and augmented realityvisualization. Memory unit 144 can provide the control processor withlocal cache memory.

Datastore 150 can include a repository of data records including useridentification records 154, bill-of-material records 156, material andlabor cost data records 158, and jurisdictional code and regulationrecords 160.

In accordance with embodiments, the signage application, data storecontent, and other components can all be local to a mobile computingdevice with sufficient on-board computing power and memory toaccommodate operations and data storage requirements. In someimplementations the components and data elements of system 100 can bedistributed between the mobile computing device, the data store, and oneor more remote servers.

An embodying method can gather end user or customer data (e.g.,including login identity, password, contact information, siteinformation, etc.). This information can be annotated or included in thesignage production package generated by the signage application forreference back to the proper customer, and building/structure details.Information relating to the end user or customer may be stored useridentification records 154, which can be accessed by the signageapplication for inclusion in the resultant design file(s).

FIG. 2 depicts process 200 for generating an augmented reality signageimage in accordance with embodiments. An image of the building/structureupon which the signage is to be installed is acquired, step 205. Signageapplication 115 can activate imaging device 117 (e.g., a camera,infrared sensor, and/or other imaging device or sensor). The capturedimage can be a still (photographic) image or a video image of thebuilding/structure.

Scaling information for the image is established, step 210. Scalingrefers to obtaining dimensional information for the physical structureand correlating the dimensional information to points on the image. Inaccordance with embodiments, scaling can be performed by one, or acombination of, several approaches. For example, (1) the image scale canbe acquired by a user inputting the building's dimensions; or (2) byusing laser measurement tool 120, which in some implementations can be acomponent of the mobile computing device and in other implementationscan be a device in communication with the mobile computing device; or(3) by utilizing mapping software in communication with the signageapplication; or (4) by calibration against an object of known size; or(5) by any combination of these and other approaches.

As an example, data from mapping software may be employed by the signageapplication to identify features (e.g., width, depth) of the buildingidentifiable from a mapping image of the building—such as a top-downview of the building. A calculation derived from the features identifiedby the application taken in combination with the mapping data canprovide the distance of the mobile device to the building. Knowing thepositional location of the mobile computing device and the positionallocation of the building can be used to derive a positional relationbetween the device and building. This positional relation can be used todetermine image perspective that can be used when scaling the image. Insome implementations, scaling information can be established byincluding within the field-of-view of the imaging device an object ofknown dimension prior to capturing the image. Inclusion of an object ofknown dimension can be used to obtain scaling factors that can beapplied to the building dimensions. This dimensionally-known object canbe a dimensionally-marked placard, measurement stick, surveyor markings,QR code target, and other items. In some implementations, machinelearning can be used to analyze the building image for features of knowndimensions (e.g., standard sized doorways, windows, etc.).

Location information of the building/structure can be obtained by thesignage application accessing position location system 116. Thisobtained positional information can be enhanced by utilizing availablecellular communication system or wireless internet (Wi-Fi) networks.

In some implementations, a user can enter dimensional data for scalingthe image. A user insert a line into the image as it is presented ondisplay 118 (using a pointer device, or touch screen capability). Theline can have two draggable endpoints that are positioned by the user tocorrespond to points on the building image, where the user knows theseparation distance. The user can then enter the separation dimension.

Data pertaining to jurisdictional signage rules (e.g., constraintsand/or requirements) can be obtained, step 215. This information can beobtained by querying remote servers, or from jurisdictional code andregulation data records 160. These signage rules can vary betweenmunicipalities, communities, private building associations, andlandlords. Jurisdictional signage rules refers broadly to laws, by-laws,rules, regulations (e.g., zoning regulation), covenants, agreements,standards, or any other legally relevant restrictions that may exist onthe use and application of signage on a building. These jurisdictionalrules or standards can be applied to the design of the sign. Forexample, restrictions can relate to whether or not a sign may be placedon a building at all, or if permitted, what sizes, shapes, colors,brightness, and/or appearance regulations may circumscribe or limit thesign.

In accordance with embodiments, signage application 115 can interrogatea remote server by supplying the building's positional coordinates,street address, zoning map identifiers, or other unique identifyingdata. Alternatively, the relevant information can be obtained frominterrogating jurisdictional code and regulation data records on thedata store.

The signage design can be created, step 220. The design can includedetails on the content of the sign (wording, image, logo, etc.) and thetype of sign (e.g., channel lettering, box sign, backlight sign, neonsign, etc.). In accordance with embodiments, the signage application cananalyze the obtained image to detect and/or identify obstacles that canobstruct the placement of the signage—e.g., windows or doors, protrudingequipment, balconies, porticos, etc. Identification of such obstaclescan preclude placement of a sign in an ill-advised location.

The signage application can include machine-learning componentsconfigured to detect such obstacles in the acquired images/video. Insome implementations, the signage application can be in communicationwith a remotely-hosted machine-learning. Such machine-learning routinesmay be configured to determine the placement or locations of signs thatare physically reasonable. In some embodiments, a user of the signageapplication can be provided an ability to confirm that the obstaclesidentified by machine-learning routine are correctly identified.

A user can enter specification data for a desired sign, such as itsdimensions, color, image content, font, brightness, combinationsthereof, etc. In some implementations the user-specified details can bestored in data store records. In some embodiments, the signageapplication can generate two- or three-dimensional models representing asign conforming to the specifications. It should be readily understoodthat multiple models representing different signs can be generated. Forexample, consideration of multiple signage located on a rooftop, above adoorway, on the building face, etc. can each have a model generated.

An augmented reality visualization of the building and sign is created,step 225. The augmented reality visualization provides the user anopportunity to view a rendering of the signage as it would appearinstalled, prior to the ordering and manufacturing of the signage. Theuser can use this opportunity to provide feedback and additional detailon the sign design during its design process.

The signage application can superimpose the signage model(s) onto theacquired building image using augmented reality. Signage application caninclude components that detect the building orientation in the acquiredimage. These components can adjust the orientation of the building inthe augmented reality rendition to reduce skew in the image relative toa viewing perspective (e.g., from off-normal to a front-facing view).Adjusting the building orientation can facilitate the augmented realityplacement of signage at a correct angle or position.

FIG. 3 depicts augmented reality visualization 300 in accordance withembodiments. Visualization 300 includes building image 310. Thisbuilding image is a real-world captured image (step 205). In someimplementations, building image 310 can be architectural or artistrenderings of the building. Superimposed on building image 310 arechannel sign 320 and box sign 325. Signs 320, 325 are designed for thestructure (step 220) and superimposed on the building image by usingaugmented reality techniques. Doorway 330 and windows 335 are identifiedobstructions (step 220).

In accordance with embodiments, the signage application can accept userinput to manipulate the image of a sign within the augmented realityvisualization as it is being displayed on display screen 118. The usercan move the sign around the design space, remove, and alter the signusing touch gestures and/or a pointing device.

In accordance with embodiments, the signage application can enhance theaugmented reality by altering its lighting, contrast (e.g., alterbrightness of the building and signage in different directions orincrements), brightness, coloring, etc. to simulate day, night, sunny orcloudy or other conditions.

At step 230 a determination is made on whether the sign design isapproved. If the design is not approved, process 200 returns to step 220to adjust the signage design. If the design is approved, process 200continues to step 235.

The signage application can generate production documentation, step 235.The production documents represent a summary of the designed signed,including data outputs that enable the physical manufacture of thedesigned signage. The signage application may generate (or cause to begenerated) a complete bill-of-materials (BOM) of the designed signage,which can be stored in BOM records 156. The BOM may comprise lists ofparts needed for manufacture of the sign, which may include raw materiallists, number and type of light sources, electrical wiring, brackets,fasteners, etc. The generated documentation can include pricingestimates, purchase orders, technical drawings, manufacturing schedulingestimates and milestones, installation details and instruction, etc.Information on material and labor costs can be obtained from materialand labor cost data records 158. Documentation can also include acontact-list of approved distributors, builders, installers, or saleschannels.

In accordance with embodiments, a user can generate productiondocumentation including all the technical drawings, BOMs, and costs foreach sign in the building design. This can be accomplished via a serverto which the signage application sends the signage designspecifications, and from which receives files containing productiondrawings and BOMs. The user/customer can browse the files and review asummary file detailing the materials required.

At step 240 a determination is made on whether the productiondocumentation is approved. If not approved, process 200 returns to step220 to adjust the signage design. If the documentation is approved,process 200 continues to step 245.

Once approved, the production documentation can be released, step 245,for fabrication of the signage. In accordance with embodiments, one ormore pieces of production documentation can be provided to a remotedesign server that can generate schematics for wiring of a lighted sign.

In accordance with some embodiments, a computer program applicationstored in non-volatile memory or computer-readable medium (e.g.,register memory, processor cache, RAM, ROM, hard drive, flash memory, CDROM, magnetic media, etc.) may include code or executable programinstructions that when executed may instruct and/or cause a controlleror processor to perform methods discussed herein such as a method forgenerating an augmented reality signage image, as disclosed above.

The computer-readable medium may be a non-transitory computer-readablemedia including all forms and types of memory and all computer-readablemedia except for a transitory, propagating signal. In oneimplementation, the non-volatile memory or computer-readable medium maybe external memory.

Although specific hardware and methods have been described herein, notethat any number of other configurations may be provided in accordancewith embodiments of the invention. Thus, while there have been shown,described, and pointed out fundamental novel features of the invention,it will be understood that various omissions, substitutions, and changesin the form and details of the illustrated embodiments, and in theiroperation, may be made by those skilled in the art without departingfrom the spirit and scope of the invention. Substitutions of elementsfrom one embodiment to another are also fully intended and contemplated.The invention is defined solely with regard to the claims appendedhereto, and equivalents of the recitations therein.

1. A method of generating an augmented reality signage solution, themethod comprising: acquiring an image of a structure (310); establishingscaling information for the structure depicted in the image; obtainingjurisdictional signage rules (160); designing a layout for signage incompliance with the jurisdictional rules, the signage to be installed onthe structure; creating an augmented reality visualization depicting theacquired image and the signage design layout; and generating productiondocumentation for the signage design layout.
 2. The method of claim 1,including: obtaining an initial approval for the signage design layoutprior to generating the production documentation; and if the initialapproval is denied, then repeating the designing and creating steps. 3.The method of claim 1, including: obtaining a final approval for thesignage design layout after generating the production documentation; ifthe final approval is denied, then repeating the designing and creatingsteps; and else releasing at least a portion of the productiondocumentation to a sign fabricator.
 4. The method of claim 1, theestablishing scaling information including correlating dimensionalinformation of the structure to points in the image.
 5. The method ofclaim 4, including obtaining the dimensional information from at leastone of user input, a laser measurement tool (120), mapping software(116), and an object of known size in the image.
 6. The method of claim1, including obtaining the jurisdictional signage rules from at leastone of a remote server (140) and a local data store (114, 150).
 7. Themethod of claim 1, designing the signage layout including: detecting thelocation of one or more obstructions (335, 330) on the structure; andincorporating the location of the one or more obstructions in thesignage layout.
 8. The method of claim 1, creating the augmented realityvisualization including adjusting the orientation of the structurewithin the acquired image to reduce skew of the structure in relation toa viewing perspective.
 9. A non-transitory computer-readable mediumcontaining executable instructions (152) that when executed by aprocessor (142) cause the processor to perform a method of generating anaugmented reality signage solution, the method comprising: acquiring animage of a structure (310); establishing scaling information for thestructure depicted in the image; obtaining jurisdictional signage rules(160); designing a layout for signage in compliance with thejurisdictional rules, the signage to be installed on the structure;creating an augmented reality visualization depicting the acquired imageand the signage design layout; and generating production documentationfor the signage design layout.
 10. The non-transitory computer-readablemedium of claim 9, the executable instructions causing the processor toperform the method, including: obtaining an initial approval for thesignage design layout prior to generating the production documentation;and if the initial approval is denied, then repeating the designing andcreating steps.
 11. The non-transitory computer-readable medium of claim9, the executable instructions causing the processor to perform themethod, including: obtaining a final approval for the signage designlayout after generating the production documentation; if the finalapproval is denied, then repeating the designing and creating steps; andelse releasing at least a portion of the production documentation to asign fabricator.
 12. The non-transitory computer-readable medium ofclaim 9, the executable instructions causing the processor to establishscaling information by correlating dimensional information of thestructure to points in the image.
 13. The non-transitorycomputer-readable medium of claim 12, the executable instructionscausing the processor to obtain the dimensional information from atleast one of user input, a laser measurement tool (120), mappingsoftware (116), and an object of known size in the image.
 14. Thenon-transitory computer-readable medium of claim 9, the executableinstructions causing the processor to obtain the jurisdictional signagerules from at least one of a remote server (140) and a local data store(114, 150).
 15. The non-transitory computer-readable medium of claim 9,the executable instructions causing the processor to obtain the signagelayout by: detecting the location of one or more obstructions (335, 330)on the structure; and incorporating the location of the one or moreobstructions in the signage layout.
 16. The non-transitorycomputer-readable medium of claim 9, the executable instructions causingthe processor to create the augmented reality visualization by adjustingthe orientation of the structure within the acquired image to reduceskew of the structure in relation to a viewing perspective.
 17. A systemfor generating an augmented reality signage solution, the systemcomprising: a mobile computing device (110) having components incommunication across an internal bus (112); the components including: asignage application (115) in communication with a control processor(111), a position locating system (116), a memory (114), and executableinstructions (113); the executable instruction when executed by theprocessor cause the processor to perform a method comprising: acquiringan image of a structure (310); establishing scaling information for thestructure depicted in the image; obtaining jurisdictional signage rules(160); designing a layout for signage in compliance with thejurisdictional rules, the signage to be installed on the structure;creating an augmented reality visualization depicting the acquired imageand the signage design layout; and generating production documentationfor the signage design layout.
 18. The system of claim 17, theexecutable instructions causing the processor to: establish scalinginformation by correlating dimensional information of the structure topoints in the image; and obtain the dimensional information from atleast one of user input, a laser measurement tool (120), mappingsoftware (116), and an object of known size in the image.
 19. The systemof claim 17, the executable instructions causing the processor to obtainthe signage layout by: detecting the location of one or moreobstructions (335, 330) on the structure; and incorporating the locationof the one or more obstructions in the signage layout.
 20. The system ofclaim 17, the executable instructions causing the processor to createthe augmented reality visualization by adjusting the orientation of thestructure within the acquired image to reduce skew of the structure inrelation to a viewing perspective.